Representations for Active Vision

As the field of Computational Vision matures, more efforts are devoted to vision systems that are active and need to interact with their envi­ ronment in real time. A prerequisite for inte­ grating Vision and Action is the development of a set of representations of the visual system's space-time, where space includes the system itself. Thus we are faced with the problem of studying the nature of appropriate representa­ tions and also with the computational task of acquiring them in a robust manner and in real time. Both of these problems are addressed in this paper from a computational point of view. In particular, we study representations needed by active visual systems in order to understand their self-motion and the structure of their en­ vironment. The representations are of l ess metric informa­ tion content than the ones traditionally used, including depth, surface normals, curvature and 3-D metric values for the parameters of rigid motion, etc.; but they are rich enough to allow the system to perform a large number of actions. These representations, indexed in image coordinates, are the direction of transla­ tion and the direction of rotation for the case of motion and a monotonic function of the depth value in the case of shape description. Their advantage comes from the fact that they can be computed from minimal and well-defined in­ put (flow or disparity values along image gradi­ ents), as opposed to the traditional ones which require image correspondence or the utilization of assumptions about the environment. If Computer Vision was once limited to the study of mappings of a given set of visual data into representa­ tions on a more abstract level, it now has become clear that Image Understanding should also include the pro­ cess of selective acquisition of data in space and time. This has led to a series of influential studies published under the headings of Active, Animate, Purposive, or Behavioral Vision. However, with a formal theory inte­ grating perception and action still lacking, most stud­ ies have treated Active Vision [Aloimonos et a/., 1988; Bajcsy, 1988; Ballard and Brown, 1992] as an extension of the classical reconstruction theory, employing activi­ ties only as a means to regularize the classical ill-posed inverse problems, in order to recover a metric represen­ tation of space-time which is general-purpose and can be used for accomplishing any task. In other words, the concept, of selective acquisition …